Rejoinder : One - Step Sparse Estimates in Nonconcave Penalized Likelihood Models

We would like to take this opportunity to thank the discussants for their thoughtful comments and encouragements on our work. The discussants raised a number of issues from theoretical as well as computational perspectives. Our rejoinder will try to provide some insights into these issues and address specific questions asked by the discussants. Most traditional variable selection criteria, such as the AIC and the BIC, are (or are asymptotically equivalent to) the penalized likelihood with the L 0 penalty, namely, p λ (|β|) = 1 2 λ 2 I(|β| = 0), and with appropriate values of λ (Fan and Li [7]). In general, the optimization of the L 0-penalized likelihood function via exhaustive search over all subset models is an NP-hard computational problem. Donoho and Huo [3] and Donoho and Elad [2] show that, under some conditions, the solution to the L 0-penalized problem can be found by solving a convex optimization problem of minimizing the L 1-norm of the coefficients, when the solution is sufficiently sparse. In other words, the NP-hard best subset variable selection can be solved by efficient convex optimization algorithms under the sparsity assumption. This sheds light on variable selection for high-dimensional models, and motivates us to use continuous penalties, such as the L 1 penalty, rather than the discontinu-ous penalties, including the L 0 penalty. The penalized likelihood procedure with the L 1 penalty coincides with LASSO. In the same spirit of LASSO, the penalized likelihood with a nonconcave penalty, such as the SCAD penalty, has been proposed for variable selection in Fan and Li [5]. LASSO and the SCAD represent the two main streams of penalization method for variable selection in the recent literature. Although both methods operate as continuous thresholding rules, they appear to be very different theoretically and